Method and device for monitoring and analyzing signals

ABSTRACT

A method and system for monitoring and analyzing at least one signal are disclosed. An abstract of at least one reference signal is generated and stored in a reference database. An abstract of a query signal to be analyzed is then generated so that the abstract of the query signal can be compared to the abstracts stored in the reference database for a match. The method and system may optionally be used to record information about the query signals, the number of matches recorded, and other useful information about the query signals. Moreover, the method by which abstracts are generated can be programmable based upon selectable criteria. The system can also be programmed with error control software so as to avoid the re-occurrence of a query signal that matches more than one signal stored in the reference database.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of application No. 13/487,119, filedJun. 1, 2012, which is a continuation of application No. 13/035,964,filed Feb. 26, 2011, which is a continuation of application Ser. No.12/655,357, filed Dec. 22, 2009, now U.S. Pat. No. 7,949,494, which is acontinuation of application Ser. No. 12/005,229, filed Dec. 26, 2007,now U.S. Pat. No. 7,660,700, which is a continuation of application Ser.No. 09/657,181, filed Sep. 7, 2000, now U.S. Pat. No. 7,346,472. Thepreviously identified patents and/or patent applications are herebyincorporated by reference, in their entireties, as if fully statedherein.

This application is related to U.S. patent application Ser. No.08/999,766, filed Jul. 23, 1997, entitled “Steganographic Method andDevice” (issued as U.S. Pat. No. 7,568,100); pending U.S. patentapplication Ser. No. 08/772,222, filed Dec. 20, 1996, entitled“Z-Transform Implementation of Digital Watermarks” (issued as U.S. Pat.No. 6,078,664); pending U.S. patent application Ser. No. 09/456,319,filed Dec. 8, 1999, entitled “Z-Transform Implementation of DigitalWatermarks” (issued as U.S. Pat. No. 6,853,726); pending U.S. patentapplication. Ser. No. 08/674,726, filed Jul. 2, 1996, entitled “ExchangeMechanisms for Digital Information Packages with BandwidthSecuritization, Multichannel Digital Watermarks, and Key Management”(issued as U.S. Pat. No. 7,362,775); pending U.S. patent applicationSer. No. 09/545,589, filed Apr. 7, 2000, entitled “Method and System forDigital[[.]] Watermarking” (issued as U.S. Pat. No. 7,007,166); pendingU.S. patent application Ser. No. 09/046,627, filed Mar. 24, 1998,entitled “Method for Combining Transfer Function with Predetermined KeyCreation” (issued as U.S. Pat. No. 6,598,162); pending U.S. patentapplication Ser. No. 09/053,628, filed Apr. 2, 1998, entitled “MultipleTransform Utilization and Application for Secure Digital Watermarking”(issued as U.S. Pat. No. 6,205,249); pending U.S. patent applicationSer. No. 09/281,279, filed Mar. 30, 1999, entitled “Optimization Methodsfor the Insertion, Protection, and Detection of Digital Watermarks inDigital Data (issued as U.S. Pat. No. 6,522,767)”; U.S. patentapplication Ser. No. 09,594,719, filed Jun. 16, 2000, entitled“Utilizing Data Reduction in Steganographic and Cryptographic Systems”(which is a continuation-in-part of PCT application No. PCT/US00/06522,filed Mar. 14, 2000, which PCT application claimed priority to U.S.Provisional Application No. 60/125,990, filed Mar. 24, 1999) (issued asU.S. Pat. No. 7,123,718); pending U.S. application Ser. No. 60/169,274,filed Dec. 7, 1999, entitled “Systems, Methods And Devices For TrustedTransactions” (issued as U.S. Pat. No. 7,159,116); and PCT ApplicationNo. PCT/US00/21189, filed Aug. 4, 2000 (which claims priority to U.S.patent application Ser. No. 60/147,134, filed Aug. 4, 1999, and to U.S.patent application Ser. No. 60/213,489, filed Jun. 23, 2000, both ofwhich are entitled, “A Secure Personal Content Server”) (issued as U.S.Pat. No. 7,475,246). The previously identified patents and/or patentapplications are hereby incorporated by reference, in their entireties,as if fully stated herein.

In addition, this application hereby incorporates by reference, as iffully stated herein, the total disclosures of U.S. Pat. No. 5,613,004“Steganographic Method and Device”; U.S. Pat. No. 5,745,569 “Method forStega-Cipher Protection of Computer Code”; and U.S. Pat. No. 5,889,868“Optimization Methods for the Insertion, Protection, and Detection ofDigital Watermarks in Digitized Data.”

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to the monitoring and analysis of digitalinformation. A method and device are described which relate to signalrecognition to enhance identification and monitoring activities.

2. Description of the Related Art

Many methods and protocols are known for transmitting data in digitalform for multimedia applications (including computer applicationsdelivered over public networks such as the internet or World Wide Web(“WWW”). These methods may include protocols for the compression ofdata, such that it may more readily and quickly be delivered overlimited bandwidth data lines. Among standard protocols for datacompression of digital files may be mentioned the MPEG compressionstandards for audio and video digital compression, promulgated by theMoving Picture Experts Group. Numerous standard reference works andpatents discuss such compression and transmission standards fordigitized information.

Digital watermarks help to authenticate the content of digitizedmultimedia information, and can also discourage piracy. Because piracyis clearly a disincentive to the digital distribution of copyrightedcontent, establishment of responsibility for copies and derivativecopies of such works is invaluable. In considering the various forms ofmultimedia content, whether “master,” stereo, NTSC video, audio tape orcompact disc, tolerance of quality will vary with individuals and affectthe underlying commercial and aesthetic value of the content. It isdesirable to tie copyrights, ownership rights, purchaser information orsome combination of these and related data into the content in such amanner that the content must undergo damage, and therefore reduction ofits value, with subsequent, unauthorized distribution, commercial orotherwise. Digital watermarks address many of these concerns. A generaldiscussion of digital watermarking as it has been applied in the art maybe found in U.S. Pat. No. 5,687,236 (whose specification is incorporatedin whole herein by reference).

Further applications of basic digital watermarking functionality havealso been developed. Examples of such applications are shown in U.S.Pat. No. 5,889,868 (whose specification is incorporated in whole hereinby reference). Such applications have been drawn, for instance, toimplementations of digital watermarks that were deemed most suited toparticular transmissions, or particular distribution and storagemediums, given the nature of digitally sampled audio, video, and othermultimedia works. There have also been developed techniques for adaptingwatermark application parameters to the individual characteristics of agiven digital sample stream, and for implementation of digitalwatermarks that are feature-based—i.e., a system in which watermarkinformation is not carried in individual samples, but is carried in therelationships between multiple samples, such as in a waveform shape. Forinstance, natural extensions may be added to digital watermarks that mayalso separate frequencies (color or audio), channels in 3D whileutilizing discreteness in feature-based encoding only known to thosewith pseudo-random keys (i.e., cryptographic keys) or possibly tools toaccess such information, which may one day exist on a quantum level.

A matter of general weakness in digital watermark technology relatesdirectly to the manner of implementation of the watermark. Manyapproaches to digital watermarking leave detection and decode controlwith the implementing party of the digital watermark, not the creator ofthe work to be protected. This weakness removes proper economicincentives for improvement of the technology. One specific form ofexploitation mostly regards efforts to obscure subsequent watermarkdetection. Others regard successful over encoding using the samewatermarking process at a subsequent time. Yet another way to performsecure digital watermark implementation is through “key-based”approaches.

SUMMARY OF THE INVENTION

A method for monitoring and analyzing at least one signal is disclosed,which method comprises the steps of: receiving at least one referencesignal to be monitored; creating an abstract of the at least onereference signal; storing the abstract of the at least one referencesignal in a reference database; receiving at least one query signal tobe analyzed; creating an abstract of the at least one query signal; andcomparing the abstract of the at least one query signal to the abstractof the at least one reference signal to determine if the abstract of theat least one query signal matches the abstract of the at least onereference signal.

A method for monitoring a plurality of reference signals is alsodisclosed, which method comprises the steps of: creating an abstract foreach one of a plurality of reference signals; storing each of theabstracts in a reference database; receiving at least one query signalto be analyzed; creating an abstract of each at least one query signal;locating an abstract in the reference database that matches the abstractof each at least one query signal; and recording the identify of thereference signal whose abstract matched the abstract of each at leastone query signal.

A computerized system for monitoring and analyzing at least one signalis also disclosed, which system comprises: a processor for creating anabstract of a signal using selectable criteria; a first input forreceiving at least one reference signal to be monitored, the first inputbeing coupled to the processor such that the processor may generate anabstract for each reference signal input to the processor; a referencedatabase, coupled to the processor, for storing abstracts of each atleast one reference signal; a second input for receiving at least onequery signal to be analyzed, the second input being coupled to theprocessor such that the processor may generate an abstract for eachquery signal; and a comparing device, coupled to the reference databaseand to the second input, for comparing an abstract of the at least onequery signal to the abstracts stored in the reference database todetermine if the abstract of the at least one query signal matches anyof the stored abstracts.

Further, an electronic system for monitoring and analyzing at least onesignal is disclosed, which system comprises: a first input for receivingat least one reference signal to be monitored, a first processor forcreating an abstract of each reference signal input to the firstprocessor through the first input; a second input for receiving at leastone query signal to be analyzed, a second processor for creating anabstract of each query signal; a reference database for storingabstracts of each at least one reference signal; and a comparing devicefor comparing an abstract of the at least one query signal to theabstracts stored in the reference database to determine if the abstractof the at least one query signal matches any of the stored abstracts.

DETAILED DESCRIPTION OF THE INVENTION

While there are many approaches to data reduction that can be utilized,a primary concern is the ability to reduce the digital signal in such amanner as to retain a “perceptual relationship” between the originalsignal and its data reduced version. This relationship may either bemathematically discernible or a result of market-dictated needs. Thepurpose is to afford a more consistent means for classifying signalsthan proprietary, related text-based approaches. A simple analogy is theway in which a forensic investigator uses a sketch artist to assist indetermining the identity of a human.

In one embodiment of the invention, the abstract of a signal may begenerated by the following steps: 1) analyze the characteristics of eachsignal in a group of audible/perceptible variations for the same signal(e.g., analyze each of five versions of the same song--which versionsmay have the same lyrics and music but which are sung by differentartists); and 2) select those characteristics which achieve or remainrelatively constant (or in other words, which have minimum variation)for each of the signals in the group. Optionally, the null case may bedefined using those characteristics which are common to each member ofthe group of versions.

Lossless and lossy compression schemes are appropriate candidates fordata reduction technologies, as are those subset of approaches that arebased on perceptual models, such as AAC, MP3, TwinVQ, JPEG, GIF, MPEG,etc. Where spectral transforms fail to assist in greater data reductionof the signal, other signal characteristics can be identified ascandidates for further data reduction. Linear predictive coding (LPC),z-transform analysis, root mean square (rms), signal to peak, may beappropriate tools to measure signal characteristics, but otherapproaches or combinations of signal characteristic analysis arecontemplated. While such signal characteristics may assist indetermining particular applications of the present invention, ageneralized approach to signal recognition is necessary to optimize thedeployment and use of the present invention.

Increasingly, valuable information is being created and stored indigital form. For example, music, photographs and motion pictures canall be stored and transmitted as a series of binary digits—1's and 0's.Digital techniques permit the original information to be duplicatedrepeatedly with perfect or near perfect accuracy, and each copy isperceived by viewers or listeners as indistinguishable from the originalsignal. Unfortunately, digital techniques also permit the information tobe easily copied without the owner's permission. While digitalrepresentations of analog waveforms may be analyzed byperceptually-based or perceptually-limited analysis it is usually costlyand time-consuming to model the processes of the highly effectiveability of humans to identify and recognize a signal. In thoseapplications where analog signals require analysis, the cost ofdigitizing the analog signal is minimal when compared to the benefits ofincreased accuracy and speed of signal analysis and monitoring when theprocesses contemplated by this invention are utilized.

The present invention relates to identification of digitally-sampledinformation, such as images, audio and video. Traditional methods ofidentification and monitoring of those signals do not rely on“perceptual quality,” but rather upon a separate and additional signal.Within this application, such signals will be called “additive signals”as they provide information about the original images, audio or video,but such information is in addition to the original signal. Onetraditional, text-based additive signal is title and author information.The title and author, for example, is information about a book, but itis in addition to the text of the book. If a book is being duplicateddigitally, the title and author could provide one means of monitoringthe number of times the text is being duplicated, for example, throughan Internet download. The present invention, however, is directed to theidentification of a digital signal--whether text, audio, or video--usingonly the digital signal itself and then monitoring the number of timesthe signal is duplicated. Reliance on an additive signal has manyshortcomings. For example, first, someone must incorporate the additivesignal within the digital data being transmitted, for example, byconcatenation or through an embedding process. Such an additive signal,however, can be easily identified and removed by one who wants toutilize the original signal without paying for its usage. If theoriginal signal itself is used to identify the content, an unauthorizeduser could not avoid payment of a royalty simply by removing theadditive signal—because there is no additive signal to remove. Hence,the present invention avoids a major disadvantage of the prior art.

One such additive signal that may be utilized is a digitalwatermark—which ideally cannot be removed without perceptually alteringthe original signal. A watermark may also be used as a monitoring signal(for example, by encoding an identifier that uniquely identifies theoriginal digital signal into which the identifier is being embedded). Adigital watermark used for monitoring is also an additive signal, andsuch a signal may make it difficult for the user who wants to duplicatea signal without paying a royalty—mainly by degrading the perceptualquality of the original signal if the watermark (and hence the additivemonitoring signal) is removed. This is, however, is a different solutionto the problem.

The present invention eliminates the need of any additive monitoringsignal because the present invention utilizes the underlying contentsignal as the identifier itself. Nevertheless, the watermark mayincrease the value of monitoring techniques by increasing the integrityof the embedded data and by indicating tampering of either the originalcontent signal or the monitoring signal. Moreover, the design of awatermarking embedding algorithm is closely related to theperceptibility of noise in any given signal and can represent an idealsubset of the original signal: the watermark bits are an inverse of thesignal to the extent that lossy compression schemes, which can be used,for instance, to optimize a watermarking embedding scheme, can yieldinformation about the extent to which a data signal can be compressedwhile holding steadfast to the design requirement that the compressedsignal maintain its perceptual relationship with the original,uncompressed signal. By describing those bits that are candidates forimperceptible embedding of watermark bits, further data reduction may beapplied on the candidate watermarks as an example of retaining a logicaland perceptible relationship with the original uncompressed signal.

Of course, the present invention may be used in conjunction withwatermarking technology (including the use of keys to accomplish securedigital watermarking), but watermarking is not necessary to practice thepresent invention. Keys for watermarking may have many forms, including:descriptions of the original carrier file formatting, mapping ofembedded data (actually imperceptible changes made to the carrier signaland referenced to the predetermined key or key pairs), assisting inestablishing the watermark message data integrity (by incorporation ofspecial one way functions in the watermark message data or key), etc.Discussions of these systems in the patents and pending patentapplications are incorporated by reference above. The “recognition” of aparticular signal or an instance of its transmission, and its monitoringare operations that may be optimized through the use of digitalwatermark analysis.

A practical difference between the two approaches of using a separate,additive monitoring signal and using the original signal itself as themonitoring signal is control. If a separate signal is used formonitoring, then the originator of the text, audio or video signal beingtransmitted and the entity doing the monitoring have to agree as to thenature of the separate signal to be used for monitoring—otherwise, theentity doing the monitoring would not know where to look, for what tolook, or how to interpret the monitoring signal once it was identifiedand detected. On the other hand, if the original signal is used itselfas a monitoring signal, then no such agreement is necessary. Moreover, amore logical and self-sufficient relationship between the original andits data-reduced abstract enhances the transparency of any resultingmonitoring efforts. The entity doing the monitoring is not looking for aseparate, additive monitoring system, and further, need not have tointerpret the content of the monitoring signal.

Monitoring implementations can be handled by robust watermark techniques(those techniques that are able to survive many signal manipulations butare not inherently “secure” for verification of a carrier signal absenta logically-related watermarking key) and forensic watermark techniques(which enable embedding of watermarks that are not able to surviveperceptible alteration of the carrier signal and thus enable detectionof tampering with the originally watermarked carrier signal). Thetechniques have obvious trade-offs between speed, performance andsecurity of the embedded watermark data.

In other disclosures, we suggest improvements and implementations thatrelate to digital watermarks in particular and embedded signaling ingeneral. A digital watermark may be used to “tag” content in a mannerthat is not humanly-perceptible, in order to ensure that the humanperception of the signal quality is maintained. Watermarking, however,must inherently alter at least one data bit of the original signal torepresent a minimal change from the original signal's “unwatermarkedstate.” The changes may affect only a bit, at the very least, or bedependent on information hiding relating to signal characteristics, suchas phase information, differences between digitized samples, root meansquare (RMS) calculations, z-transform analysis, or similar signalcharacteristic category.

There are weaknesses in using digital watermark technology formonitoring purposes. One weakness relates directly to the way in whichwatermarks are implemented. Often, the persons responsible for encodingand decoding the digital watermark are not the creator of the valuablework to be protected. As such, the creator has no input on the placementof the monitoring signal within the valuable work being protected.Hence, if a user wishing to avoid payment of the royalty can find a wayto decode or remove the watermark, or at least the monitoring signalembedded in the watermark, then the unauthorized user may successfullyduplicate the signal with impunity. This could occur, for example, ifeither of the persons responsible for encoding or decoding were to havetheir security compromised such that the encoding or decoding algorithmswere discovered by the unauthorized user.

With the present invention, no such disadvantages exist because thecreator need not rely on anyone to insert a monitoring signal—as no suchsignal is necessary. Instead, the creator's work itself is used as themonitoring signal. Accordingly, the value in the signal will have astrong relationship with its recognizability.

By way of improving methods for efficient monitoring as well aseffective confirmation of the identity of a digitally-sampled signal,the present invention describes useful methods for using digital signalprocessing for benchmarking a novel basis for differencing signals withbinary data comparisons. These techniques may be complemented withperceptual techniques, but are intended to leverage the generallydecreasing cost of bandwidth and signal processing power in an age ofincreasing availability and exchange of digitized binary data.

So long as there exist computationally inexpensive ways of identifyingan entire signal with some fractional representation or relationshipwith the original signal, or its perceptually observable representation,we envision methods for faster and more accurate auditing of signals asthey are played, distributed or otherwise shared amongst providers(transmitters) and consumers (receivers). The ability to massivelycompress a signal to its Essence—which is not strictly equivalent to“lossy” or “lossless” compression schemes or perceptual codingtechniques, but designed to preserve some underlying “aesthetic quality”of the signal—represents a useful means for signal analysis in a widevariety of applications. The signal analysis, however, must maintain theability to distinguish the perceptual quality of the signals beingcompared. For example, a method which analyzed a portion of a song bycompressing it to a single line of lyrics fails to maintain the abilityto distinguish the perceptual quality of the songs being compared.Specifically, for example, if the song “New York State of Mind” werecompressed to the lyrics “I'm in a New York State of Mind,” such acompression fails to maintain the ability to distinguish between thevarious recorded versions of the song, say, for example between BillyJoel's recording and Barbara Streisand's recording. Such a method is,therefore, incapable of providing accurate monitoring of the artist'srecordings because it could not determine which of the two artists isdeserving of a royalty—unless of course, there is a separate monitoringsignal to provide the name of the artist or other information sufficientto distinguish the two versions. The present invention, however, aims tomaintain some level of perceptual quality of the signals being comparedand would deem such a compression to be excessive.

This analogy can be made clearer if it is understood that there are alarge number of approaches to compressing a signal to, say, 1/10,000thof its original size, not for maintaining its signal quality to ensurecomputational ease for commercial quality distribution, but to assist inidentification, analysis or monitoring of the signal. Most compressionis either lossy or lossless and is designed with psychoacoustic orpsychovisual parameters. That is to say, the signal is compressed toretain what is “humanly-perceptible.” As long as the compressionsuccessfully mimics human perception, data space may be saved when thecompressed file is compared to the uncompressed or original file. Whilepsychoacoustic and psychovisual compression has some relevance to thepresent invention, additional data reduction or massive compression isanticipated by the present invention. It is anticipated that theoriginal signal may be compressed to create a realistic or self-similarrepresentation of the original signal, so that the compressed signal canbe referenced at a subsequent time as unique binary data that hascomputational relevance to the original signal. Depending on theapplication, general data reduction of the original signal can be assimple as massive compression or may relate to the watermark encodingenvelope parameter (those bits which a watermarking encoding algorithmdeem as candidate bits for mapping independent data or those bits deemedimperceptible to human senses but detectable to a watermark detectionalgorithm). In this manner, certain media which are commonly known bysignal characteristics, a painting, a song, a TV commercial, a dialect,etc., may be analyzed more accurately, and perhaps, more efficientlythan a text-based descriptor of the signal. So long as the sender andreceiver agree that the data representation is accurate, even insofar asthe data-reduction technique has logical relationships with theperceptibility of the original signal, as they must with commonly agreedto text descriptors, no independent cataloging is necessary.

The present invention generally contemplates a signal recognition systemthat has at least five elements. The actual number of elements may varydepending on the number of domains in which a signal resides (forexample, audio is at least one domain while visual carriers are at leasttwo dimensional). The present invention contemplates that the number ofelements will be sufficient to effectively and efficiently meet thedemands of various classes of signal recognition. The design of thesignal recognition that may be used with data reduction is betterunderstood in the context of the general requirements of a pattern orsignal recognition system.

The first element is the reference database, which contains informationabout a plurality of potential signals that will be monitored. In oneform, the reference database would contain digital copies of originalworks of art as they are recorded by the various artists, for example,contain digital copies of all songs that will be played by a particularradio station. In another form, the reference database would contain notperfect digital copies of original works of art, but digital copies ofabstracted works of art, for example, contain digital copies of allsongs that have been preprocessed such that the copies represent theperceptual characteristics of the original songs. In another form, thereference database would contain digital copies of processed data files,which files represent works of art that have been preprocessed in such afashion as to identify those perceptual differences that candifferentiate one version of a work of art from another version of thesame work of art, such as two or more versions of the same song, but bydifferent artists. These examples have obvious application to visuallycommunicated works such as images, trademarks or photographs, and videoas well.

The second element is the object locator, which is able to segment aportion of a signal being monitored for analysis (i.e., the “monitoredsignal”). The segmented portion is also referred to as an “object.” Assuch, the signal being monitored may be thought of comprising a set ofobjects. A song recording, for example, can be thought of as having amultitude of objects. The objects need not be of uniform length, size,or content, but merely be a sample of the signal being monitored.Visually communicated informational signals have related objects; colorand size are examples.

The third element is the feature selector, which is able to analyze aselected object and identify perceptual features of the object that canbe used to uniquely describe the selected object. Ideally, the featureselector can identify all, or nearly all, of the perceptual qualities ofthe object that differentiate it from a similarly selected object ofother signals. Simply, a feature selector has a direct relationship withthe perceptibility of features commonly observed. Counterfeiting is anactivity which specifically seeks out features to misrepresent theauthenticity of any given object. Highly granular, and arguablysuccessful, counterfeiting is typically sought for objects that areeasily recognizable and valuable, for example, currency, stamps, andtrademarked or copyrighted works and objects that have value to a bodypolitic.

The fourth element is the comparing device which is able to compare theselected object using the features selected by the feature selector tothe plurality of signals in the reference database to identify which ofthe signals matches the monitored signal. Depending upon how theinformation of the plurality of signals is stored in the referencedatabase and depending upon the available computational capacity (e.g.,speed and efficiency), the exact nature of the comparison will vary. Forexample, the comparing device may compare the selected object directlyto the signal information stored in the database. Alternatively, thecomparing device may need to process the signal information stored inthe database using input from the feature selector and then compare theselected object to the processed signal information. Alternatively, thecomparing device may need to process the selected object using inputfrom the feature selector and then compare the processed selected objectto the signal information. Alternatively, the comparing device may needto process the signal information stored in the database using inputfrom the feature selector, process the selected object using input fromthe feature selector, and then compare the processed selected object tothe processed signal information.

The fifth element is the recorder which records information about thenumber of times a given signal is analyzed and detected. The recordermay comprise a database which keeps track of the number of times a song,image, or a movie has been played, or may generate a serial output whichcan be subsequently processed to determine the total number of timesvarious signals have been detected.

Other elements may be added to the system or incorporated into the fiveelements identified above. For example, an error handler may beincorporated into the comparing device. If the comparing deviceidentifies multiple signals which appear to contain the object beingsought for analysis or monitoring, the error handler may offer furtherprocessing in order to identify additional qualities or features in theselected object such that only one of the set of captured signals isfound to contain the further analyzed selected object that actuallyconforms with the object thought to have been transmitted ordistributed.

Moreover, one or more of the five identified elements may be implementedwith software that runs on the same processor, or which uses multipleprocessors. In addition, the elements may incorporate dynamic approachesthat utilize stochastic, heuristic, or experience-based adjustments torefine the signal analysis being conducted within the system, including,for example, the signal analyses being performed within the featureselector and the comparing device. This additional analyses may beviewed as filters that are designed to meet the expectations of accuracyor speed for any intended application.

Since maintenance of original signal quality is not required by thepresent invention, increased efficiencies in processing andidentification of signals can be achieved. The present inventionconcerns itself with perceptible relationships only to the extent thatefficiencies can be achieved both in accuracy and speed with enablinglogical relationships between an original signal and its abstract.

The challenge is to maximize the ability to sufficiently compress asignal to both retain its relationship with the original signal whilereducing the data overhead to enable more efficient analysis, archivingand monitoring of these signals. In some cases, data reduction alonewill not suffice: the sender and receiver must agree to the accuracy ofthe recognition. In other cases, agreement will actually depend on athird party who authored or created the signal in question. A digitizedsignal may have parameters to assist in establishing more accurateidentification, for example, a “signal abstract” which naturally, or byagreement with the creator, the copyright owner or other interestedparties, can be used to describe the original signal. By utilizing lessthan the original signal, a computationally inexpensive means ofidentification can be used. As long as a realistic set of conditions canbe arrived at governing the relationship between a signal and its datareduced abstract, increases in effective monitoring and transparency ofinformation data flow across communications channels is likely toresult. This feature is significant in that it represents an improvementover how a digitally-sampled signal can be cataloged and identified,though the use of a means that is specifically selected based upon thestrengths of a general computing device and the economic needs of aparticular market for the digitized information data being monitored.The additional benefit is a more open means to uniformly catalog,analyze, and monitor signals. As well, such benefits can exist for thirdparties, who have a significant interest in the signal but are not thesender or receiver of said information.

As a general improvement over the art, the present inventionincorporates what could best be described as “computer-acoustic” and“computer-visual” modeling, where the signal abstracts are created usingdata reduction techniques to determine the smallest amount of data, atleast a single bit, which can represent and differentiate two digitizedsignal representations for a given predefined signal set. Each of suchrepresentations must have at least a one bit difference with all othermembers of the database to differentiate each such representation fromthe others in the database. The predefined signal set is the objectbeing analyzed. The signal identifier/detector should receive itsparameters from a database engine. The engine will identify thosecharacteristics (for example, the differences) that can be used todistinguish one digital signal from all other digital signals that arestored in its collection. For those digital signals or objects which areseemingly identical, except[ing] that the signal may have differentperformance or utilization in the newly created object, benefits overadditive or text-based identifiers are achieved. Additionally, decisionsregarding the success or failure of an accurate detection of any givenobject may be flexibly implemented or changed to reflect market-baseddemands of the engine. Appropriate examples are songs or works or artwhich have been sampled or reproduced by others who are not the originalcreator.

In some cases, the engine will also consider the NULL case for ageneralized item not in its database, or perhaps in situations wheredata objects may have collisions. For some applications, the NULL caseis not necessary, thus making the whole system faster. For instance,databases which have fewer repetitions of objects or those systems whichare intended to recognize signals with time constraints or capture alldata objects. Greater efficiency in processing a relational database canbe obtained because the rules for comparison are selected for themaximum efficiency of the processing hardware and/or software, whetheror not the processing is based on psychoacoustic or psychovisual models.The benefits of massive data reduction, flexibility in constructingappropriate signal recognition protocols and incorporation ofcryptographic techniques to further add accuracy and confidence in thesystem are clearly improvements over the art. For example, where thedata reduced abstract needs to have further uniqueness, a hash orsignature may be required. And for objects which have further uniquenessrequirements, two identical instances of the object could be made uniquewith cryptographic techniques.

Accuracy in processing and identification may be increased by using oneor more of the following fidelity evaluation functions:

1) RMS (root mean square). For example, a RMS function may be used toassist in determining the distance between data based on mathematicallydeterminable Euclidean distance between the beginning and end datapoints (bits) of a particular signal carrier.

2) Frequency weighted RMS. For example, different weights may be appliedto different frequency components of the carrier signal before usingRMS. This selective weighting can assist in further distinguishing thedistance between beginning and end points of the signal carrier (at agiven point in time, described as bandwidth, or the number of total bitsthat can be transmitted per second) and may be considered to be themathematical equivalent of passing a carrier signal difference through adata filter and figuring the average power in the output carrier.

3) Absolute error criteria, including particularly the NULL set(described above) The NULL may be utilized in two significant cases:First, in instances where the recognized, signal appears to be anidentified object which is inaccurately attributed or identified to anobject not handled by the database of objects; and second, where acollision of data occurs. For instance, if an artist releases a secondperformance of a previously recorded song, and the two performances areso similar that their differences are almost imperceptible, then thepreviously selected criteria may not be able to differentiate the tworecordings. Hence, the database must be “recalibrated” to be able todifferentiate these two versions. Similarly, if the system identifiesnot one, but two or more, matches for a particular search, then thedatabase may need “recalibration” to further differentiate the twoobjects stored in the database.

4) Cognitive Identification. For example, the present invention may usean experience-based analysis within a recognition engine. Once suchanalysis may involve mathematically determining a spectral transform orits equivalent of the carrier signal. A spectral transform enablessignal processing and should maintain, for certain applications, somecognitive or perceptual relationship with the original analog waveform.As a novel feature to the present invention, additional classes may besubject to humanly-perceptible observation. For instance, anexperience-based criteria which relates particularly to the envisionedor perceived accuracy of the data information object as it is used orapplied in a particular market, product, or implementation. This mayinclude a short 3 second segment of a commercially available andrecognizable song which is used for commercials to enable recognition ofthe good or service being marketed. The complete song is marketed as aseparately valued object from the use of a discrete segment of the song(that may be used for promotion or marketing—for the complete song orfor an entirely different good or service). To the extent that an ownerof the song in question is able to further enable value through thelicensing or agreement for use of a segment of the original signal,cognitive identification is a form of filtering to enabledifferentiations between different and intended uses of the same orsubset of the same signal (object). The implementation relatingspecifically, as disclosed herein, to the predetermined identificationor recognition means and/or any specified relationship with subsequentuse of the identification means can be used to create a history as tohow often a particular signal is misidentified, which history can thenbe used to optimize identification of that signal in the future. Thedifference between use of an excerpt of the song to promote a separateand distinct good or service and use of the excerpt to promoterecognition of the song itself (for example, by the artist to sellcopies of the song) relates informationally to a decision based onrecognized and approved use of the song. Both the song and applicationsof the song in its entirety or as a subset are typically based onagreement by the creator and the sender who seeks to utilize the work.Trust in the means for identification, which can be weighted in thepresent invention (for example, by adjusting bit-addressableinformation), is an important factor in adjusting the monitoring orrecognition features of the object or carrier signal, and by using anymisidentification information, (including any experience-based orheuristic information), additional features of the monitored signal canbe used to improve the performance of the monitoring system envisionedherein. The issue of central concern with cognitive identification is agreater understanding of the parameters by which any given object is tobe analyzed. To the extent that a creator chooses varying and separateapplication of his object, those applications having a cognitivedifference in a signal recognition sense (e.g., the whole or anexcerpt), the system contemplated herein includes rules for governingthe application of bit-addressable information to increase the accuracyof the database.

5) Finally, the predetermined parameters that are associated with adiscrete case for any given object will have a significant impact uponthe ability to accurately process and identify the signals. For example,if a song is transmitted over a FM carrier, then one skilled in the artwill appreciate that the FM signal has a predetermined bandwidth whichis different from the bandwidth of the original recording, and differenteven from song when played on an AM carrier, and different yet from asong played using an 8-bit Internet broadcast. Recognition of thesedifferences, however, will permit the selection of an identificationmeans which can be optimized for monitoring a FM broadcasted signal. Inother words, the discreteness intended by the sender is limited anddirected by the fidelity of the transmission means. Objects may becataloged and assessing with the understanding that all monitoring willoccur using a specific transmission fidelity. For example, a databasemay be optimized with the understanding that only AM broadcast signalswill be monitored. For maximum efficiency, different data bases may becreated for different transmission channels, e.g., AM broadcasts, FMbroadcasts, Internet broadcasts, etc.

For more information on increasing efficiencies for information systems,see The Mathematical Theory of Communication (1948), by Shannon.

Because bandwidth (which in the digital domain is equated to the totalnumber of bits that can be transmitted in a fixed period of time) is alimited resource which places limitations upon transmission capacity andinformation coding schemes, the importance of monitoring for informationobjects transmitted over any given channel must take into considerationthe nature and utilization of a given channel. The supply and demand ofbandwidth will have a dramatic impact on the transmission, andultimately, upon the decision to monitor and recognize signals. Adiscussion of this is found in a co-pending application by the inventorunder U.S. patent application Ser. No. 08/674,726 (which issued Apr. 22,2008 as U.S. Pat. No. 7,362,775) “Exchange Mechanisms for DigitalInformation Packages with Bandwidth Securitization, Multichannel DigitalWatermarks, and Key Management” (which application is incorporatedherein by reference as if fully setforth herein).

If a filter is to be used in connection with the recognition ormonitoring engine, it may be desirable for the filter to anticipate andtake into consideration the following factors, which affect theeconomics of the transmission as they relate to triggers for paymentand/or relate to events requiring audits of the objects which are beingtransmitted: 1) time of transmission (i.e., the point in time when thetransmission occurred), including whether the transmission is of a liveperformance); 2) location of transmission (e.g., what channel was usedfor transmission, which usually determines the associated cost for usageof the transmission channel); 3) the point of origination of thetransmission (which may be the same for a signal carrier over manydistinct channels); and 4) pre-existence of the information carriersignal (pre-recorded or newly created information carrier signal, whichmay require differentiation in certain markets or instances).

In the case of predetermined carrier signals (those which have beenrecorded and stored for subsequent use), “positional information carriersignals” are contemplated by this invention, namely, perceptualdifferences between the seemingly “same” information carrier that can berecognized as consumers of information seek different versions orquality levels of the same carrier signal. Perceptual differences existbetween a song and its reproduction from a CD, an AM radio, and anInternet broadcast. To the extent that the creator or consumer of thesignal can define a difference in any of the four criteria above, meanscan be derived (and programmed for selectability) to recognize anddistinguish these differences. It is, however, quite possible that theability to monitor carrier signal transmission with these factors willincrease the variety and richness of available carrier signals toexisting communications channels. The differentiation between anabsolute case for transmission of an object, which is a time dependentevent, for instance a live or real time broadcast, versus the relativecase, which is prerecorded or stored for transmission at a later pointin time, creates recognizable differences for signal monitoring.

The monitoring and analysis contemplated by this invention may have avariety of purposes, including, for example, the following: to determinethe number of times a song is broadcast on a particular radio broadcastor Internet site; to control security though a voice-activated securitysystem; and to identify associations between a beginner's drawing andthose of great artists (for example to draw comparisons betweentechnique, compositions, or color schemes). None of these examples couldbe achieved with any significant degree of accuracy using a text-basedanalysis. Additionally, strictly text-based systems fail to fullycapture the inherent value of the data recognition or monitoringinformation itself.

SAMPLE EMBODIMENTS Sample Embodiment 1

A database of audio signals (e.g., songs) is stored or maintained by aradio station or Internet streaming company, who may select a subset ofthe songs are stored so that the subset may be later broadcast tolisteners. The subset, for example, may comprise a sufficient number ofsongs to fill 24 hours of music programming (between 300 or 500 songs).Traditionally, monitoring is accomplished by embedding some identifierinto the signal, or affixing the identifier to the signal, for lateranalysis and determination of royalty payments. Most of the traditionalanalysis is performed by actual persons who use play lists and otherstatistical approximations of audio play, including for example, dataobtained through the manual (i.e., by persons) monitoring of astatistically significant sample of stations and transmission times sothat an extrapolation may be made to a larger number of comparablemarkets.

The present invention creates a second database from the first database,wherein each of the stored audio signals in the first database is datareduced in a manner that is not likely to reflect the human perceptualquality of the signal, meaning that a significantly data-reduced signalis not likely to be played back and recognized as the original signal.As a result of the data reduction, the size of the second database (asmeasured in digital terms) is much smaller than the size of the firstdatabase, and is determined by the rate of compression. If, for example,if 24 hours worth of audio signals are compressed at a 10,000:1compression rate, the reduced data could occupy a little more than 1megabyte of data. With such a large compression rate, the data to becompared and/or analyzed may become computationally small such thatcomputational speed and efficiency are significantly improved.

With greater compression rates, it is anticipated that similarity mayexist between the data compressed abstractions of different analogsignals (e.g., recordings by two different artists of the same song).The present invention contemplates the use of bit-addressabledifferences to distinguish between such cases. In applications where thedata to be analyzed has higher value in some predetermined sense,cryptographic protocols, such as a hash or digital signature, can beused to distinguish such close cases.

In a preferred embodiment, the present invention may utilize acentralized database where copies of new recordings may be deposited toensure that copyright owners, who authorize transmission or use of theirrecordings by others, can independently verify that the object iscorrectly monitored. The rules for the creator himself to enter his workwould differ from a universally recognized number assigned by anindependent authority (say, ISRC, ISBN for recordings and booksrespectively). Those skilled in the art of algorithmic informationtheory (AIT) can recognize that it is now possible to describe optimizeduse of binary data for content and functionality. The differencesbetween objects must relate to decisions made by the user of the data,introducing subjective or cognitive decisions to the design of thecontemplated invention as described above. To the extent that objectscan have an optimized data size when compared with other objects for anygiven set of objects, the algorithms for data reduction would havepredetermined flexibility directly related to computational efficiencyand the set of objects to be monitored. The flexibility in havingtransparent determination of unique signal abstracts, as opposed toindependent third party assignment, is likely to increase confidence inthe monitoring effort by the owners of the original signals themselves.The prior art allows for no such transparency to the copyright creators.

Sample Embodiment 2

Another embodiment of the invention relates to visual images, which ofcourse, involve at least two dimensions.

Similar to the goals of a psychoacoustic model, a psychovisual modelattempts to represent a visual image with less data, and yet preservethose perceptual qualities that permit a human to recognize the originalvisual image. Using the very same techniques described above inconnection with an audio signal, signal monitoring of visual images maybe implemented.

One such application for monitoring and analyzing visual images involvesa desire to find works of other artists that relate to a particulartheme. For example, finding paintings of sunsets or sunrises. Atraditional approach might involve a textual search involving a databasewherein the works of other artists have been described in writing. Thepresent invention, however, involves the scanning of an image involvinga sun, compressing the data to its essential characteristics (i.e.,those perceptual characteristics related to the sun) and then findingmatches in a database of other visual images (stored as compressed oreven uncompressed data). By studying the work of other artists usingsuch techniques, a novice, for example, could learn much by comparingthe presentations of a common theme by different artists.

Another useful application involving this type of monitoring andanalyzing is the identification of photographs of potential suspectswhose identity matches the sketch of a police artist.

Note that combinations of the monitoring techniques discussed above canbe used for audio-visual monitoring, such as video-transmission by atelevision station or cable station. The techniques would have tocompensate, for example, for a cable station that is broadcasting aaudio channel unaccompanied by video.

Other embodiments and uses of the invention will be apparent to thoseskilled in the art from consideration of the specification and practiceof the invention disclosed herein. The specification and examples shouldbe considered exemplary only with the true scope and spirit of theinvention indicated by the following claims. As will be easilyunderstood by those of ordinary skill in the art, variations andmodifications of each of the disclosed embodiments can be easily madewithin the scope of this invention as defined by the following claims.

1. A method for monitoring and analyzing at least one signal comprising:receiving at least one reference signal to be monitored; creating anabstract of said at least one reference signal; storing the abstract ofsaid at least one reference signal in a reference database; receiving atleast one query signal to be analyzed; creating an abstract of said atleast one query signal; comparing the abstract of said at least onequery signal to the abstract of said at least one reference signal todetermine if the abstract of said at least one query signal matches theabstract of said at least one reference signal. 2-100. (canceled)